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Terminal  Air  Traffic  Control  Laboratory  System 
in  Operation 

Handovers  of  inbound  and  outbound  aircraft  are  coordinated  between  the 
local  controllers  (foreground)  and  the  conversion  controllers  (background) 
by  the  traffic  coordinator  (not  shown). 
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TEST  RESULTS  OF  THE 

TERMINAL  AIR  TRAFFIC  CONTROL  LABORATORY  SYSTEM 


by  A.  S.  Cooperband 
L.  T.  Alexander 
H.  S.  Schmitz 


1.  INTRODUCTION 

System  design  is  still  an  art.  As  an  art,  the  processes  of 
conceptualizing  a  system,  choosing  the  component  pieces,  and  putting  them 
together  depend  upon  the  experience  and  intuition  of  a  group  of  experts. 

If  there  is  ever  to  be  a  science  of  systems,  one  of  the  prerequisites  is  a 
methodology  through  which  system  phenomena  can  be  reproduced  in  the  laboratory 
for  scientific  study. 

The  Terminal  Air  Traffic  Control  (TATC)  project  was  based  on  the  premise 
that  significant  facts  ceui  be  discovered  about  how  systems  operate  by  putting 
an  entire  system*  in  the  laboratory,  simulating  a  task  environment  with 
which  it  would  interact,  and  watching  it  operate.  In  this  technique,  at 
first,  there  is  no  systematic  attempt  to  abstract  either  the  system  components 
or  the  system  operations.  Instead,  we  chose  to  study  first  the  problem  of 
how  to  design  a  computer-based  terminal  air  traffic  control  system.  By 
designing  and  manipulating  a  laboratory  model  of  such  a  system  we  hoped  to 
uncover  not  only  problems  of  information  transmission  between  men  and 
computers  in  an  operational  situation,  but  also  the  attendant  problems  of 
simulating  real-time,  man-computer  systems  rapidly  and  with  least  cost. 

2.  TEST  OBJECTIVES 


At  the  outset  of  the  TATC  project,  a  minimal  system  consisting  of  the 
control  agencies  at  one  airport  terminal  was  modeled  in  System  Development 
Corporation's  Systems  Simulation  Research  Laboratory.  This  initial  system 


*  The  problem  of  what  constitutes  an  adequate  system  for  study  by  this 
method  is  a  matter  for  much  argument.  In  this  context  we  are  reminded  of 
an  observation,  attributed  to  Dr.  Milton  Weiner  of  RAND,  that  "one  man's 
system  is  another  man's  subsystem."  Techniques  have  been  devised  for 
embedding  the  laboratory  experimental  system  in  a  larger  organizational 
context  with  which  it  interacts.  We  do  not  intend  to  address  the  problem 
of  >Aiat  the  size  of  the  experimental  system  should  be  except  to  provide  a 
cookbook  formula,  viz.,  as  large  as  possible  subject  to  the  capability  of 
the  experimenter  to  control  the  environment  within  ;diich  the  system  operates 
as  constrained  by  laboratory  space  and  computer  capacity. 
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served  primarily  as  an  exploratory  device.  Some  of  the  system  design 
objectives  and  a  description  of  the  system  are  presented  in  TM -639/ 002/ 00. 
The  present  paper  discusses  the  results  of  the  system  shakedown  tests 
performed  during  the  summer  of  I962. 

After  computer  program  testing  was  completed  in  the  spring  of  19^2,  a 
live  shakedown  and  familiarization  test  was  run.  This  test,  in  addition  to 
extending  the  program  checkout  to  the  more  complex  live  situation,  was 
intended  to  familiaxize  the  system  designers  and  researchers  with  the  system 
by  letting  them  observe  it  in  operation  under  a  variety  of  environmental 
conditions.  To  complement  this  "naturalistic"  observation,  three  general 
questions  were  posed;  (l)  What  is  the  capability  of  the  system  to  control 
air  traffic?  (2)  How  sensitive  is  the  system  to  a  variety  of  traffic  and 
procedural  conditions?  (3)  Do  the  computer  programs  or  hardware  restrict 
the  performance  of  the  operators? 

3.  THE  LABORATORY  SYSTEM 

The  laboratory  system  was  modeled  after  the  San  Francisco -Oakland 
geographic  area.  This  geographic  area  measures  100  by  130  nautical  miles 
and  extends  nautical  miles  high.  Within  this  volume,  the  airport  was 
responsible  for  two  cylinders  of  airspace  called  the  conversion  zone  and 
the  local  eurea  (see  Figure  l). 

A  distinction  was  made  between  the  "test  system"  and  the  "embedding 
system."  The  test  system  consisted  of  two  subsystems;  conversion  control 
and  local  control.  The  conversion  control  subsystem,  consisting  of  two 
controllers,  was  responsbile  for  all  aircraft  in  transition  between  the 
en  route  phase  and  the  final  approach  or  initial  departure  phases.  The 
local  control  subsystem,  consisting  of  a  traffic  coordinator,  and  two  other 
controllers,  was  responsible  for  approaches  and  departures.  Communications 
between  these  subsystems  were  via  a  flight  data  processor,  the  ccmputer. 

It  is  this  test  system  •vdiich  was  manipulated  and  analyzed.  The  embedding 
system  consisted  of  surroionding  agencies  with  •vdiich  the  test  system 
communicated.  It  included  two  ground  controllers,  two  sector  controllers, 
and  up  to  seven  pilots.  The  environment  of  the  test  system  was  controlled 
through  the  embedding  system. 
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4.  TECT  PROCEDURES 

4.1.  Selection  of  Subjects 

Three  crews  were  used  during  the  summer  shakedown  test.  Crews  A  and  B 
were  each  composed  of  six  male  undergraduate  students  from  local  colleges  and 
universities.  These  were  the  test  crews.  A  third  crew.  Crew  C,  also 
composed  of  male  undergraduate  students,  operated  as  pilots  and  controllers  in 
the  embedding  system.  Probability  of  attendance  throughout  the  study  was 
encouraged  by  dividing  hourly  pay  into  two  parts:  base  pay  and  a  bonus  -vdiich 
could  be  received  only  for  perfect  attendance. 

The  subjects  in  Crews  A  aund  B  were  chosen  with  reasonably  clear  speech, 
normal  hearing  and  vision,  an  Intelligence  test  score  in  the  range  of  plus 
or  minus  one  sigma  from  mean  score  for  college  freshmen,  and  a  lack  of 
knowledge  of  air  traffic  control.  The  members  of  Crew  C  were  not  selected 
according  to  any  special  criteria. 

4. 2.  Training  Procedures 

The  embedding  system.  Including  Crew  C,  was  trained  over  a  three -month 
period  preceding  the  system  test.  The  training  program  for  the  test  subjects 
began  with  a  one-week  orientation  course.  After  con^detion  of  the  orientation, 
the  group  was  divided  into  two  crews,  matched  on  the  basis  of  ACE  ( Language )- 
centile  scores,  an  achievement  test  designed  to  measure  knowledge  acquired 
about  the  system  during  the  orientation  course,  and  the  opinions  of  the 
experimenters  who  served  as  instructors.  Each  crew  member  was  assigned  a 
control  position  which  he  kept  through  the  entire  training  and  test  series. 

Crews  were  assigned  to  one  of  two  training  treatments  by  the  flip  of 
a  coin.  Crew  A  was  trained  under  on-the-job  conditions;  Crew  B  was  given 
"schematic  training."  In  schematic  training  a  set  of  schematic  drawings 
of  the  face  of  each  console  was  used.  Colored  pegs  simulated  the  illumi¬ 
nation  of  various  console  lights,  and  magnetized  rectangular  plaques  were 
moved  on  a  large  magnetic  blackboard  to  simulate  the  GEOPLAN  (air  situation 
display)  with  aircraft  moving  through  the  system.  The  two  test  crews  were 
given  the  same  training  problems.  For  the  schematic  training  crew,  the 
instructions  moved  the  magnetized  plaques  across  the  blackboard  in  non-real 
time  so  as  to  follow  the  same  flight  paths  that  were  presented  in  the  "live" 
training  problem. 
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4.3.  Training  Problems 

Except  for  the  first  problem,  •vAilch  was  a  one -hour  problem,  each 
training  problem  lasted  two  hours.  The  problems  were  divided  into  30-minute 
segments,  each  designed  to  present  at  least  two  or  three  air  situations 
■vdiich  stressed  at  least  one  important  control  procedure,  e.g.,  an  emergency 
procedvire,  a  change  in  flight  plan  procedvire,  or  a  handover  procedure.  In 
addition,  the  problems  were  designed  to  greidually  increase  traffic  load, 
rate  of  input,  and  traffic  mix  or  heterogeneity  (i.e.,  aircraft  -vdilch 
differed  markedly  in  performance).  Altogether,  training  and  orientation 
lasted  five  weeks. 

4.4.  Test  Design 

The  system  was  tested  with  24  two-hour  problems  involving  a  total  of 
960  aircrarft.  The  test  was  designed  as  a  factorial  combination  of  four 
variables.  Three  of  the  variables  describe  the  traffic  characteristics;  the 
fourth  was  procedural. 

1.  Input  Rate:  the  rate  at  >rtilch  inbourid  aircraft  penetrate  the 
system.  Two  values  were  used:  (l)  uniform  and  (2)  non-uniform. 
For  the  tmiform  rate,  aircraft  were  scheduled  to  enter  the  system 
about  every  2-1/ 2  minutes  over  a  13 -minute  interval;  for  the  non- 
\miform  rate,  an  equivalent  number  of  inbcrunds  were  scheduled  to 
penetrate  the  system  in  a  7 -minute  interval,  with  two -thirds  of 
the  aircraft  penetrating  during  the  middle  4-minutes.  The  number 
of  outbounds  was  held  constant. 

2.  Distribution:  the  geographic  distribution  of  the  inbound  aircraft 
between  the  two  conversion  zones  (see  Figure  l).  Three  values  were 
used:  (1)  equal  distribution,  (2)  north  greater,  (3)  south 
greater.  Where  the  distribution  was  not  equal,  one  zone  received 
twice  as  many  aircraft  as  the  other. 

3.  Composition:  this  refers  to  the  performance  characteristics  of 
the  aircraft  in  the  traffic  sample.*  Two  values  were  used: 

(1)  heterogeneous,  (2)  homogeneous.  In  the  homogeneous  case,  all 
aircraft  were  hi^  performance,  such  as  a  commercial  jet;  in  the 
heterogeneous  case,  the  aircraft  were  evenly  divided  between 
medium  performance,  such  as  a  DC-7,  and  supersonic. 


*  These  data  were  obtained  from  L.  Farr  and  H.  Schmitz,  "An  Estimate  of  the 
1970-1975  Environment  for  Air  Traffic  Control,"  iM-599/OOO/Ol. 
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4.  Configiiration;  the  two  procedures  followed  by  the  crews-- 

configurations  I  and  IIo  In  configuration  I  the  flight  plans 
for  an  aircraft  included  at  least  three  fixes  between  the  point 
at  which  the  system  was  penetrated  and  the  airport;  in  configu¬ 
ration  II,  no  intermediate  fixes  were  specified.  Also,  separation 
standards  in  the  local  area  were  lower  for  configirration  II  than 
for  configuration  I. 

The  variables  of  input  rate  and  distribution  represented  uniform  and 
non-uniform  dispersion  of  aircraft  in  time  and  In  space;  the- composition 
variable  represented  uniform  and  non-uniform  dispersions  of  system  transition 
times.  The  configuration  varie^sle  required  directed  or  opportunistic 
responses  of  the  system. 

Each  combination  of  variables  gave  rise  to  a  traffic  schedule  for  one 
"problem  period."  A  problem  period  vas  designed  to  last  30  minutes.  During 
that  time,  six  inbound  aind  four  outbound  flights  were  scheduled.  A  flight 
remained  within  the  system  for  about  I5  minutes  on  the  average.  The  first 
12  problem  periods,  corresponding  to  the  following  cells  In  the  factorial 
design,  were  under  configuration  I,  as  in  Figure  2. 


Order 

Input  Rate 

Distribution 

Composition 

Configuration 

1 

1 

1 

2 

I 

2 

2 

2 

1 

I 

3 

1 

3 

2 

I 

4 

2 

2 

2 

I 

5 

2 

3 

1 

I 

6 

1 

2 

2 

I 

7 

1 

2 

I 

8 

1 

1 

I 

9 

3 

1 

I 

10 

3 

2 

I 

11 

1 

1 

I 

12 

2 

I 

Fig.  2. --Factorial  combination  of  variables 
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The  next  12  replicated  the  first  12.  The  last  24  repeated  the  first  2k  except 
that  configuration  II  was  used  instead  of  configuration  I.  Each  of  the  two 
crews  was  presented  with  the  same  sequence  of  problem  periods.  A  crew  ran 
for  approximately  two  hours  every  other  day,  a  run  consisting  of  four  problem 
periods. 

4. 5.  Problem  Design 


Twelve  standard  sets  of  flight  paths  were  generated,  one  for  each  of 
the  unique  combinations  of  the  traffic  variables.  In  preparing  the  schedule 
for  a  problem  period,  these  flight  paths  were  translated  into  flight  plans 
according  to  the  starting  time  of  the  particular  problem  period.  Since  psurt 
of  the  system  design  was  based  on  the  assvimptlon  of  a  central  scheduling 
function  which  guaranteed  flight  plans  to  he  free  of  conflict  at  fixes,  the 
standard  flight  paths  were  processed  by  a  computer  progrsoa  Which  resolved 
such  conflicts  by  revising  altitude  assignments.  The  output  from  this  conflict 
resolution  program  was  examined  manually  and  adjusted  further  idiere  necessary 
to  conform  as  closely  as  poselble  to  the  standaurd  paths.  Then  flight -plan 
strips  were  printed  autcmatlcally  for  the  subjects  end  for  simulators, 
observers,  and  experimenters.  Certain  parameters  of  these  flight  plans  were 
used  by  another  computer  program  to  produce  a  control  deck  of  punched  cards 
which  supplied  all  the  necessaury  infocnaation  Viich  the  computer  programs  in 
the  test  and.  embedding  systems  needed  to  "create"  these  flights. 

4.6.  Data  Collection 

Observation  auid  data  collection  were  both  direct  and  indirect.  The 
embedding  system  operated  on  a  balcony  surrounding  the  test  room  and  separated 
from  it  by  one-way  glass.  This  permitted  direct  visuaj.  observation  of  the 
test  crews.  In  addition,  direct  observation  of  communications  was  possible 
by  means  of  microphones  in  the  test  room  and  hy  means  of  monitor  speakers 
connected  to  the  simulated  radio  system  and  intercom  system.  Indirect 
observations  were  made  from  duplicates  of  the  displays  used  by  the  test 
system  and  fran  deductions  based  on  information  displayed  to  pilots  and 
the  ground  and  sector  control  simulators.  Similarly,  the  direct  data 
collection  consisted  of  such  items  as  tape  recordings  of  conversations 
and  discussions,  polygraph  recordings  of  selected  switch  actions,  and  notes 
taken  about  interesting  or  \inusiial  situations.  The  indirect  data  collection 
was  based  upon  a  record  that  was  kept  by  the  computer  of  every  switch  action 
taken,  as  well  as  selected  tables  of  information.  These  recordings  were  made 
every  few  seconds.  From  the  magnetic  tape  recordings  of  the  switch  actions, 
the  entire  run  could  be  re-created  by  the  computer  program,  playing  back  the 
taped  switch  actions  instead  of  using  live  switch  insertions.  Aircraft 
histories  were  constructed  directly  from  the  recorded  selected  tables;  these 
histories  formed  the  basis  for  deducing  further  measures  of  system  performance. 
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4.7-  Conduct  of  a  Test  Run 


A  test  run  started  with  &  briefing  of  the  embedding  system  by  a 
problem  designer.  Concvirrently,  the  test  crew  was  holding  its  own  briefing, 
discussing  the  flight  etrip*  for  the  first  segment  of  the  day's  run.  Next, 
the  embedding  system  personnel  worked  with  the  computer  to  check  out  all 
equipment  components  used  for  man-computer  communication;  they  also  checked 
the  radio  and  the  Intercom  systems.  After  they  had  ascertained  that  all 
equipment  was  operating  properly,  the  embedding  system  personnel  took  their 
positions  and  the  test  crew  was  ushered  Into  the  test  room.  As  soon  as 
everyone  was  ready,  the  synchronized  clocke  were  started  end  the  run  began. 
When  the  last  aircraft  froa  the  last  problem  period  was  out  of  the  test 
system,  the  run  was  terminated  by  a  phone  call  frem  the  supervisor  (an 
experimenter).  This  was  followed  by  an  embedding  system  debriefing  and 
a  conciirrent  test  crew  debriefing.  The  test  crew  debriefing  was  in  two 
parts.  In  the  first  part,  which  was  unsupervised,  the  crew  discussed  their 
performance  In  the  run  ^st  completed.  In  the  second  part,  led  by  a 
researcher,  the  crew  received  a  report  of  their  errors,  which  they  discussed. 
The  researcher  discussed  procedures  with  them  and  tried  to  get  them  to 
evaluate  the  system  design  end  make  suggestions.  Procedural  natters  which 
came  up  in  one  crew's  debriefing  were  Introduced  In  the  other  crew's 
debriefing.  The  researcher  also  acted  as  sn  Intermediary  between  the  crew 
and  the  simulated  administrative  organization  constructed  above  it. 

5.  MEASURES 

There  is  no  single  measure  by  which,  system  performance  can  be  evaluated. 
Instead,  four  general  categories  of  measures  seem  to  be  significant  in  air 
traffic  control. 

5.1.  Categories  of  Measures 

Safety.  One  of  the  prime  reqpilrements  of  the  system  is  that  it  control 
air  traffic  safely.  Two  aspects  of  safety  were  measurable:  the  number  of 
violations  of  prescribed  separation  standetrds  between  aircraft  and  the 
probability  that  a  collision  would  occur  in  the  immediate  future  between  any 
pair  of  aircraft. 

Expeditiousness .  This  is  a  broad  category  with  many  aspects.  The 
performance  of  the  system  in  expediting  the  tretffic  flow  was  measured  by: 
the  per  cent  of  flight  time  spent  holding*;  the  per  cent  of  controlled 


*  Holding  an  aircraft  refers  to  instructing  it  to  fly  in  an  orbit  around 
a  designated  point.  This  represents  a  delay  in  proceeding  to  its  destination. 
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aircraft  "vAiich  are  held;  the  delay  imposed  on  the  traffic  as  compared  to  a 
theoretical  optimum  flow;  the  ratio  of  average  flight  time  to  the  average 
theoretical  flight  time;  the  average  time  in  buffer;  and  the  average 
transition  time. 

Orderliness.  Within  the  context  of  an  air  traffic  control  system, 
order  is  a  difficult  concept  to  pin  down.  As  an  end  in  itself,  orderliness 
cannot  be  defended.  But  to  the  extent  that  it  tends  to  increase  safety, 
expeditiousness,  and  economy,  it  improves  the  performance  of  the  system. 

One  aspect  of  orderliness  seems  particularly  significant:  the  smoothing  of 
the  variations  in  arrival  times  at  the  approach  buffer.  Since  the  maximum 
acceptsuice  capability  of  a  runway  can  be  realized  only  with  a  flow  of  traffic 
regularly  spaced  in  time  at  the  approach  gates,  with  spacing  equal  to  the 
minimum  rvinway  service  time,  it  is  advantageous  to  regulate  the  traffic  to 
conform  to  this  pattern.  The  meastire  of  the  regulation  of  the  traffic  flow 
is  called  "smoothing." 

Other  Measures.  We  were  also  interested  in  developing  measures  of  the 
cost  of  system  operation,  or  the  efficiency  with  which  resources  were 
utilized.  The  amount  of  fuel  consumed  by  an  aircraft  -vdiile  \inder  control 
of  the  TATC  system  is  such  a  measvire. 

5. 2.  Description  of  Individual  Measures 

The  separate  meastires  caui  be  described  as  follows: 

Collision  Probability.  The  probability,  expressed  as  a  per  cent,  that 
a  collision  will  occur  while  aircraft  are  under  control  of  the  specified 
system  or  subsystem. 

Safety  Violations.  The  number  of  times  two  or  more  aircraft  flew 
closer  than  the  specified  separation  limits. 

Time  Held.  The  per  cent  of  total  time  that  aircraft  under  system 
control  spent  holding. 

Aircraft  Held.  The  per  cent  of  aircraft  under  system  control  which 
were  held  at  least  once. 

Schedule  Delay.  The  average  delay  imposed  on  traffic  by  the  control 
system;  the  difference  between  actual  flight  time  and  a  theoretical  time 
required  to  fly  the  shortest  available  path  from  the  point  of  entering  upon 
system  control  to  the  point  of  leaving  it.  Negative  values  are  possible. 
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Schedule  Ratio.  The  average  ratio  of  actueO.  to  theoretical  flight 

times. 


Mean  Spacing.  The  average  time  betveen  the  arrival  of  aircraft  at 
designated  points  in  the  terminal  area.  In  general,  a  lower  value  means 
better  performance. 

Time  in  Buffer.  The  average  time  that  an  aircraft  spent  waiting  to  be 
transferred  from  the  conversion  control  to  the  local  control  subsystem. 

Transition  Time.  The  average  time  during  which  an  Inbound  aircraft 
was  the  responsibility  of  the  conversion  control  subsystem  but  was  not  being 
held  in  the  buffer. 

Smoothing.  The  variability  of  differences  in  arrival  times  at 
designated  points  in  the  terminal  area  relative  to  a  theoretical  uncontrolled 
traffic  flow.  Negative  values  are  possible;  a  more  positive  value  means 
better  performance . 

Fuel  Consumption.  The  cost  of  fuel  consumed  by  an  aircraft  while  under 
system  control .  • 

6.  RESULTS 


In  addition  to  design  verification  and  program  checkout,  the  test 
was  intended  to  allow  researchers  to  observe  the  system  and  to  produce 
some  information  about  the  system's  ability  to  control  air  traffic,  about 
the  sensitivity  of  the  system  to  varying  traffic  and  procedviral  conditions, 
and  about  the  interaction  of  the  operators  axid  the  compruter  via  the  hardware 
Interface.  To  this  end,  measures  of  system  and  subsystem  performance  were 
applied.  They  showed  some  interesting  relationships  about  the  way  the  crews 
and  system  reacted  to  the  varying  environmental  conditions. 

Tables  1  through  4  summarize  the  results  of  the  system  test.  In  these 
tables,  the  effect  of  each  of  the  variables  is  evaluated  with  the  measures 
of  system  performance  described  in  the  preceding  section. 

6.1.  Crew  Differences 


Subjective  evaluation  of  the  two  crews  throughout  the  test  runs  showed 
both  inter-  and  intra-crew  differences.  In  terms  of  the  conversion  control 
subsystem  and  the  local  control,  subsystem,  these  differences  can  be  summarized 
as  follows: 
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1.  The  conversion  controllers  of  Crew  A  were  slightly  superior  to 
those  of  Crew  B.  There  was  a  greater  disparity  in -performance 
between  controllers  in  Crew  B  than  in  Crew  A. 

2.  In  both  crews  the  individual  local  controllers  were  roughly 
canparablej  the  local  control  team  of  Crew  A  was  somewhat  better 
than  that  of  Crew  B. 

3*  The  traffic  coordinator  of  Crew  A  Integrated  the  performance  of 
the  other  crew  members;  the  traffic  coordinator  of  Crew  B  was 
relatively  ineffectual. 

From  these  subjective  observations  of  crsv  characteristics  we  predicted 
that  Crew  A  wooild  perform  better  than  Crew  S;  that  Inter -crew  conversion 
control  differences  should  be  less  than  local  control  differences,  which  in 
tiirn  should  be  less  than  total  system  differences. 

Table  1  shows  how  the  two  crews  performed.  Tour  points  are  evident 
from  this  table; 

1.  The  performance  of  the  crews  differed  significantly  only  as  regards 
the  expeditious  handling  of  traffic. 

2.  There  is  no  significant  difference  In  performance  between  the 
conversion  control  subsystem  of  the  two  crews. 

3.  Crew  B  performed  better  than  Crew  X  In  only  2  of  the  23  measures 
in  the  table. 

4.  On  the  average,  aircraft  were  hell  in  the  buffers  about  1. 5  times 
longer  by  Crew  B  than  Crew  A.  For  arriving  aircraft,  the  buffers 
are  the  interface  between  ths  two  systems  in  which  aircraft  form 

a  queue;  the  conversion  control  subsystem  puts  aircraft  in  and  the 
local  control,  subsystem  removes  them.  The  average  time  that  air¬ 
craft  are  held  in  the  buffers  is  a  good  measure  of  the  integration 
of  the  subsystems. 

These  resTilts  support  the  subjective  evaluation  of  the  crews .discussed 
previously.  It  is  interesting  that,  in  the  analysis  of  variance  performed 
on  the  data,  there  were  no  statistically  significant  interactions  involving 
the  crew  variable. 


The  significance  of  the  differences  betveen  the  means  shown  was  determined  by  an  analysis 
of  varieince. 
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The  significance  of  the  differences  between  the  means  shown  and  determined  by  an  analysis 
of  variance. 
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The  significance  of  the  differences  between  the  means  shown  was  determined  by  ein  analysis 
of  variance. 
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6.2.  Traffic  Variables 


Air  traffic  control  consists  of  three  phases;  the  control  of  traffic 
on  the  ground  at  an  airport;  the  control  of  traffic  en  route  from  one 
airport  complex  to  another;  and  the  control  of  traffic  within  the  airport 
complex  between  the  en  route  phase  and  the  ground  phase.  The  latter  phase, 
designated  terminal  control,  consists  of  two  major  functions:  the  control 
of  aircraft  in  transition  between  the  relatively  stable  conditions  of  en 
route  flights  and  the  final  phases  of  landing  or  Initial  phases  of  takeoff; 
and  the  control  of  aircraft  in  the  immediate  vicinity  of  the  runways. 

The  mission  of  a  terminal  air  traffic  control  system  is  to  make  the 
most  efficient  use  of  the  airspace  under  its  jurisdiction  so  as  to  provide 
for  the  needs  of  eill  the  potential  users  of  that  airspace,  at  the  same  time 
Insviring  the  safe  separation  of  the  aircraft  under  its  control.  Efficient 
use  of  airspace  means  that  aircraft  will  be  scheduled  over  the  shortest 
routes  available,  in  as  rapid  and  orderly  a  way  as  possible,  and  so  that 
the  cost  of  flying  (mainly  fuel  consumption)  will  be  minimized.  The 
orderliness  of  control  has  to  do  with  optimsQ.  sequencing  of  aircraft  with 
widely  differing  performance  characteristics,  from  high-speed  military  or 
commercial  jets  to  single-engine  sport  planes  and  heliocopters.  All  classes 
of  flyers  have  a  right  to  efficient  direction  and  control  -from  the  system. 

Of  all  the  criteria  of  system  proficiency,  safety  is  the  most  Important 
In  the  existing  air  traffic  control  system,  the  Federal. Aviation  Agency 
has  established  strict  standards  for  maintaining  sepeuration  in  time  and 
space  between  aircraft.  Unfortunately  the  criteria  of  safety,  expedltiousnes 
and  orderliness  of  traffic  handling,  and  the  democratization  of  the  airspace 
are  not  all  compatible.  The  subjects  were  instructed  to  achieve  the 
prescribed  safety  standards  and  only  within  these  limits  to  optimize  the 
other  criteria. 

If  we  ignore  the  crew  veuriable  in  considering  the  effect  of  the  traffic 
veuriables*  on  system  performance  in  general  we  see,  in  Tables  2  and  3>  that 
the  system  was  relatively  insensitive  to  variations  in  either  distribution 
or  composition  of  traffic.  Again,  most  of  the  differences  in  performance 
occur  in  expeditious  handling  of  traffic,  and  primeurlly  by  the  conversion 
control  subsystem. 


*  In  practice  the  different  input  rates  proved  indistinguishable.  As  a 
result,  the  problem  periods  representing  different  values  of  this  variable 
were  analyzed  as  replications  of  the  other  variables. 
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There  is  a  striking  similarity  betveen  the  effects  of  these  variables 
on  the  performance  of  the  local  control  subsystem  and  the  total  system.  If 
the  local  control  subsystem  performs  veil  on  a  particular  measure  under  a 
particular  traffic  configurauion,  the  total  system  responds  similarly.  If 
ve  consider  the  measures  on  Tables  2  and  3#  we  find  that  this  effect  occurs 
18  out  of  22  times.  This  suggests,  that  in  the  TATC  system  the  local  control 
subsystem  made  the  major  contribution  to  total  system  performeince.  This 
result  is  surprising  if  one  considers  the  limited  amount  of  airspace 
controlled  by  the  local  control  subsystem,  the  relatively  shorter  time, 
compared  to  the  conversion  control  subsystem,  during  vhich  it  exercised 
control  over  each  aircraft,  and  the  relatively  small  number  of  control 
options  available  to  the  local  control  subsystem. 

6.3«  Scheduling  Variable 

One  of  the  most  interesting  results  from  the  standpoint  of  future 
investigations  vas  due  to  the  effects  of  the  scheduling  variable.  Systems 
\4iich  command  or  control  the  actions  of  other  systems  usually  make  schedtles 
or  plans  for  the  disposition  and  utilization  of  the  resources  of  the 
controlled  system.  Military  commanders  make  var  plans,  business  managers 
make  production  schedules,  etc.  One  important  aspect  of  these  plans  is 
that  they  make  explicit  a  sequence  of  events  vhich  are  to  occur  at  particular 
times. 


For  the  air  traffic  control  system  this  sequence  of  events  is  the 
traffic  schedule  by  means  of  vhich  the  control  system  inspires  the  safe, 
expeditious  and  orderly  handling  of  traffic.  Traffic  schedtles  are  composed 
of  flight  plans  which  indicate  the  predicted  locations  of  aircraft  at 
partictlar  times  in  the  future.  The  essence  of  air  traffic  control  is  the 
establishment  and  maintenance  of  schedules.  To  perform  this  function,  the 
control  system  should  be  able  to  accept  all  aircraft  input  to  it,  assign  a 
flight  plan  to  each,  monitor'how  well  each  aircraft  conforms  to  its  schedule, 
and,  if  deviations  occur  because  of  either  drift  or  emergency  conditions,  to 
readjust  the  schedule. 

The  scheduling  variable  was  introduced  in  an  attempt  to  determine 
•viiether  a  controller  could  schedule  and  control  traffic  in  the  terminal 
area  without  benefit  of  complete  fli^t  plans.  The  variable  consisted  of 
two  sets  of  operating  rules  idiich  differed  in  the  degree  of  forma4-  scheduling 
of  the  air  traffic  ■vdiich  was  imposed  on  the  controllers.  Under  one  set  of 
rules,  the  "pre-planned"  condition,  the  controllers  were  provided  with  a 
complete  flight  plan  for  each  aircraft,  and  no  path  deviation  was  permitted. 
Under  the  other  set  of  rules,  the  "flexible "condition,  only  the  entry  and 
exit  points  in  the  terminal,  complex  were  given,  and  the  operators  had  to 
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organize  the  traffic  themselves  by  scheduling  and  routing  aircraft  according 
to  the  exigencies  of  the  situation  as  it  changed  from  moment  to  moments 

The  operating  rules  for  flexibility  of  scheduling  were  presumed  to 
affect  only  the  conversion  control  subsystem,  although  the  effects  of  this 
variable  upon  total  system  and  local  control  subsystem  performance  are  also 
shown  in  Table  4®  However,  only  the  performance  of  the  conversion  control 
subsystem  will  be  discussed « 

The  results  indicate  that  when  the  conversion  control  subsystem  was 
able  to  schedule  the  routes  of  aircraft  according  to  flexible  flight 
scheduling  rules,  it  handled  the  traffic  more  expeditiously  than  when  it 
had  to  conform  to  fixed  schedule  flight  ruleso  Under  fixed  scheduling  rules, 
4  per  cent  of  the  aircraft  spent  time  holding  as  compared  to  1  per  cent  under 
flexible  scheduling  rules;  under  fixed  rules  36  per  cent  of  the  aircraft 
were  held  at  any  time  as  compared  to  12  per  cent  under  flexible  rules <.  These 
reductions  are  both  significant  and  occurred  with  no  compromise  in  safety. 
Although  the  collision  probability  measure  showed  an  increase,  the  number  of 
violations  were  reduced.  Also,  the  traffic  seemed  to  be  controlled  in  a  more 
orderly  fashion. 

As  far  as  schedule  accommodation  is  concerned,  however,  this  effect  is 
reversed.  Under  flexible  scheduling  rules,  approximately  11  seconds  per 
aircraft  is  added  to  the  minimum  schedule;  this  shows  up  also  in  the  schedule 
ratio.  An  Increase  of  9  seconds  in  the  average  transition  time  per  aircraft 
provides  an  additional  clue  concerning  the  apparent  contradiction  in  these 
results. 

To  maintain  adequate  separation  between  aircraft  and  still  impose 
little  delay  on  their  progress,  the  controllers  seem  to  have  assigned  them 
to  routes  which  were  slightly  longer  than  the. most  direct  path.  In  other 
words,  to  accommodate  the  exigencies  of  the  traffic  environment,  the 
controllers  were  trading  space  for  time. 

6.4.  Contoller-Computer  Interaction 

One  of  the  objectives  of  the  TATC  test  was  to  uncover  and  study 
problems  involved  in  the  interaction  between  the  operator  and  the  computer 
while  performing  air  traffic  control  operations.  The  initial  system  configu¬ 
ration  was  intended  to  assist  the  controller  in  doing  his  job  by  automating 
some  of  his  functions.  These  functions  varied  among  the  different  positions 
but  we  can  identify  four  general  areas  of  computer  assistance:  display 
generation;  status  and  record  keeping;  conflict  prediction;  and  monitoring. 
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Our  observations  of  the  system  during  the  test  led  to  the  subjective 
conclusion  that,  although  the  computer  did  provide  sane  degree  of  assistance, 
in  general  the  amount  of  information  vhich  the  controller  had  to  insert 
exceeded  the  information  -which  he  received  in  return.  At  the  conclusion  of 
the  test,  an  analysis  of  the  conversion  control  subsystem  led  to  the  estimate 
that  the  conversion  controller  supplied  about  4  times  as  much  information  as 
he  received,  or,  stated  another  -way,  he  received  back  about  25  per  cent  as 
much  information  as  he  supplied. 

Whether  this  relationship  materially  affected  the  capability  of  the 
operator  to  control  air  traffic  could  not  be  determined  from  these  preliminary 
test  runs.  The  measure  of  information  interchange  seems  to  be  too  simple  a 
criterion  upon  -vdiich  to  evaluate  such  questions  of  allocation  of  function, 
organization  and  display  of  information,  ar.d  requirements  for  information 
insertion  by  the  himian.  These  are  matters  for  future  investigation.  If 
there  are  to  be  humans  in  a  control  system,  a  primary  question  seems  to  be 
how  to  reduce  their  operating  load  so  that  they  can  perform  their  assigned 
fvinctions  with  the  degree  of  proficiency  required  to  achieve  the  system 
mission. 

7.  DISCUSSION 

The  TATC  project  represents  an  attempt  to  apply  and  evaluate  a  particu¬ 
lar  model  for  studying  the  operations  of  a  class  of  systems  usually  identified 
as  command-control,  information-processing,  and  decision-making. 

Such  systems  characteristically  operate  in  non-deterministic  or  emergent 
environments.  They  charactistically  accept  and  process  large  OTiounts  of 
rapidly  changing  data,  make  decisions  based  on  these  data,  and  issue  command 
instructions  -vdiich  modify  a  controllable  environment  -while  conforming  to  a 
complex  set  of  criteria.  Generally  these  systems  include  computers,  people, 
and  mechanisms  by  -which  they  can  communicate  -within  the  system  and  with  the 
outside  world.  Their  successful  design  and  operation  usually  depends  upon 
the  solution  of  problems  of  information  transmission  bet-ween  the  computer 
and  the  man. 

These  systems  usually  are  so  leirge  and  complex  that  it  is  difficult 
to  study  their  operations  under  real -life  conditions.  The  classical  scien¬ 
tific  technique  in  such  a  situation  is  to  choose  one  aspect  of  system 
operations,  small  enough  to  be  controllable  in  the  laboratory,  and  study  it. 
The  method  used  in  the  TATC  project  is  different.  It  seeks  to  reproduce  in 
the  laboratory  a  large  portion  of  the  system  consisting  of  interrelated 
functions,  to  generate  an  environment  -vdiich  is  as  similar  as  possible  to 
the  real-life  system  environment,  and  to  study  the  operations  of  the 
laboratory  system  in  that  environment. 
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7»1.  Was  the  Test  a  Success? 

The  objectives  of  the  system  test  were  limited:  to  enable  the 
researchers  to  gain  familiarity  with  the  system  by  observing  it  and  to 
provide  information  which  could  be  used  for  redesign  purposes.  We  wanted 
to  find  out  whether  the  initial  configuration  we  had  designed  would  work: 
whether  it  could  actually  control  air  traffic  in  a  simiolated  environment, 
whether  its  performance  would  be  affected  by  certain  traffic  conditions 
and  procedural  modifications,  and  whether  problems  of  infonaation  interchange 
between  the  computer  programs  and  the  operators  could  be  identified.  With 
regard  to  these  alms,  the  test  was  successful. 

However,  the  test  of  an  initial  laboratory  system  configuration  was 
part  of  a  larger  research  project  with  broader  objectives;  to  learn  how 
to  design  and  simulate  systems, in  the  laboratory  and  to  identify  questions 
for  future  research.  The  remainder  of  the  discussion  will  deal  with  these 
objectives. 

7. 2.  Design  and  Simulation 


The  design  of  a  laboratory  system  as  complex  as  the  TATC  System 
presents  most,  if  not  all,  of  the  prob.lems  which  must  be  solved  by  the 
designer  of  a  real  system  operating  in  the  field.  The  development  of  this 
system  provided  a  good  exercise  in  the  art  of  system  design  which  might 
well  be  expanded  to  provide  a  useful  training  device  for  designers.  The 
cost  of  making  errors  under  these  conditions  is  extremely  small  in  compaxison 
to  the  cost  of  similar  errors  in  designing  real  systems. 

One  of  the  most  important  problems  identified  dTxring  the  design  and 
development  process  was  the  difficulty  of  communication  among  engineers, 
programmers,  and  human  factors  personnel.  To  alleviate  this  situation  we 
developed  an  inexpensive  and  rapid  method  of  simulating  the  operations 
of  the  desired  system  and,  in  a  series  of  games,  "played  throu^"  all  of 
the  functions  which  the  controllers  and  the  computer  were  required  to 
perform  together.  This  technique  provided  a  common  referent  experience 
and  was  used  to  generate  a  common  task  language  which  facilitated  communi¬ 
cation  among  the  participants.  This  technique,  ^diich  we  call  "schematic 
simulation,"  is  discussed  in  more  detail  in  TM-639/OO5/OO. 

We  found  that  it  was  extremely  difficult  to  standardize  and  control 
the  activities  of  the  people  who  operated  the  embedding  system.  They 
introduced  too  much  unpredictable  variability  in  the  system  environment, 
thus  making  it  difficult  to  reproduce  planned  stimulus  conditions  from  nan 
to  run.  To  the  extent  that  rules  can  be  written  for  the  operation  of  the 
embedding  agencies,  it  seems  desirable  to  automate  these  functions.  The 
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resulting  decrease  in  flexibility  in  providing  inputs  to  the  laboratory 
system  is  compensated  for  by  the  achievement  of  greater  control  over 
environmental  conditions. 

We  also  produced  an  Important  methodological  tool  for  studying  complex 
laboratory  systems;  the  technique  of  data  regeneration.  During  the  operation 
of  the  system^,  a  set  of  initial  conditions  is  reccrded.  Then  a  complete 
record  is  made  of  all  keyboard  interventions.  Since  the  environment  was 
controlled  directly  or  indirectly  only  through  these  keyboard  interventions, 
they  constitute  a  complete  and  unsmbiguous  record  of  the  performance  of  the 
system.  At  a  later  time,  these  recorded  keyboard  insertions  can  be  "played 
back"  through  the  original  system  computer  programs  to  regenerate  all  of  the 
data  that  existed  during  the  operations. 

7.3«  Research  Questions 


By  operating  the  TATC  system  in  the  laboratory,  we  were  able  to 
identify  several  questions  for  future  research.  The  results  show  that  the 
markedly  different  ability  of  each  crew  to  operate  as  a  team  affected  the 
performance  of  the  system;  that  system  performance  was  relatively  insensitive 
to  the  variations  in  traffic  conditions,  and  that  effective  air  traffic 
control  could  be  conducted  under  flexible  scheduling  rules.  What  was  there 
about  the  design  of  the  system  and  its  operation  •vdiich  could  account  for 
these  results? 

The  experimenters  predicted  that  Crew  A  would  perform  better  than 
Crew  B,  and  in  general  this  prediction  was  upheld,  although  the  differences 
in  performance  reach  the  accepted  level  of  statistical  significance  only 
for  the  expeditious  handling  of  traffic.  It  is  no  surprise  that  the  better 
team  was  able  to  operate  the  system  better.  However,  we  may  conjecture  that 
this  phenomenon  may  occur  only  in  systems  •vdiich  are  so  designed  as  to  make 
maximal  use  of  the  adaptive  capabilities  of  the  personnel  subsystem;  that  is, 
in  systems  which  are  designed  so  that  people  can  use  the  equipment  and 
programs  in  new  and  different  ways  to  suit  the  changing  demands  of  the 
environment.  One  of  the  ways  to  do  this  is  to  provide  the  operators  with 
information  by  which  they  can  predict  or  recognize  the  effect  of  their 
actions  upon  the  performance  of  other  operators  and  upon  total  system 
performance.  When  a  system  is  designed  in  this  way,  "functional  visibility" 
is  increased.*  A  question  for  future  research  might  be;  What  is  the  effect 
of  functional  visibility  on  crew  performance? 


*  Functional  visibility  in  systems  is  described  by  L.  Alexander,  et.al. 
"The  effectiveness  of  knowledge  of  results  in  a  military  system-training 
program,"  J.  Appl.  Psychol.,  I962,  202-211. 
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The  results  also  showed  that  the  operators  could  use  the  system 
adaptively  and  could  conduct  air  traffic  control  under  flexible  scheduling 
rules.  Flexible  scheduling  procedures  were  introduced  because  we  felt  that 
rigid  scheduling  of  flight  plans  might  be  restrictive  in  the  relatively 
confined  airspace  available  in  the  terminal  area.  We  wanted  to  see  if  the 
operators  could  use  the  system  to  adapt  flight  schedules  to  the  traffic 
exigencies  'vdiich  might  arise.  There  are  three  aspects  of  scheduling  which 
will  have  to  be  considered:  the  generation  of  schedules;  the  monitoring 
of  the  air  traffic  to  predict  and  detect  potentially  serious  deviations 
from  schedules;  and  the  resolxition  of  these  deviations  by  schedule  readjust¬ 
ment.  We  do  not  propose  that  the  entire  scheduling  function  should  be  done 
by  the  humans  in  the  systemj,  but  the  finding  that  the  system  could  operate 
under  flexible  scheduling  procedures  opens  the  door  to  a  systematic 
investigation  of  the  question:  How  can  the  capabilities  of  the  man  and  the 
computer  be  combined  to  perform  a  complex  scheduling  function? 

System  performance  was  relatively  insensitive  to  variations  in  the 
geographic  distribution  of  traffic  and  in  the  heterogeneity  of  the  traffic 
samples.  This  may  have  resulted  from  the  ability  of  the  crews  to  adapt 
their  control  procedures  easily  to  the  traffic  variations  However,  a 
simpler  question  should  be  tested  first:  Were  these  results  due  to  a 
relatively  low  traffic  load  and  to  a  restricted  range  of  values  of  the  traffic 
variables? 

The  methods  used  in  the  TATC  project  constitute  a  relatively  new 
technique  in  system  research.  Certainly  these  methods  eire  not  as  precise  as 
those  of  classical  research.  Even  so,  the  laboratory  technique  we  have  used 
seems  to  be  the  only  one  available  for  controlled  study  of  intact  man-machine 
informat ion -processing  systems.  In  order  to  achieve  precision,  new  experi¬ 
mental  design  and  statistical  analysis  tools  will  have  to  be  developed. 

There  seem  to  be  four  main  problems  involved  in  the  development  and 
application  of  this  method  for  studying  systems:  How  does  one  simulate, 
manipulate,  and  control  the  complex  environments  of  laboratory  systems? 

How  does  one  collect  meaningful  data  regarding  the  performance  of  the 
system  as  a  whole  and  especially  as  regards  how  system  performance  relates 
to  the  performance  of  the  subsystem  and  individual  components?  How  does 
one  quickly  and  economically  modify  the  component  parts,  their  functions  and 
interrelations  so  that  a  wide  variety  of  configurations  can  be  examined? 

And  would  the  coordinate  use  of  computer  models  assist  in  the  development  of 
theories  of  system  performance  and  in  the  testing  of  hypotheses  derived  from 
these  theories? 
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8.  SUMMARY 


The  TATC  project  represents  an  attempt  to  produce  in  the  laboratory  the 
phenomena  which  occur  in  man-machine,  information-processing  systems.  The 
essential  aspect  of  the  procedure  is  to  create  the  complex  environment  within 
which  such  systems  operate,  to  manipulate  this  environment,  and  to  observe 
how  the  system  responds  in  an  effort  to  achieve  its  mission.  The  instrviment 
which  makes  this  technique  possible  is  the  high-speed  digital  computer,  for 
only  throu^  its  use  can  these  complex  environments  be  simulated,  manipulated, 
and  controlled. 

We  believe,  on  the  basis  of  our  experience  with  TATC,  that  significant 
system  phenomena  can  be  studied  in  the  laboratory  thereby  contributing  to 
the  development  of  a  science  of  systems. 


UNCLASSIFIED 


System  Development  Coip)oratlon, 

Santa  Monica;,  California 
TEST  RESULTS  OF  THE  TERMINAL  AIR 
TRAFFIC  CONTROL  LAEORATORY  SYSTEM. 
Scientific  rept.^  by 

A.  S.  Cooperband,  L.  T.  Alescander, 

E.  S.  Schmitz.  ,  2.3  September  1963# 

27p.>  2  figs.,  4  tables 

Unclassified  report 

DESCRIPTORS s  Mr  Tr'affie  Control  Systems. 

States  that  the  TATC  (Termi.iial  Mr 
Traffic  Control)  project  attempts  to 
produce  in  the  laboratory  the  phenomena 


■which  occ\ir  in  man^machine,  information- 
processing  systems.  Reports  that  the 
essential  aspect  of  the  procedure  is  to 
create  the  complex  environment  within 
which  such  systems  operate,  to  manipulate 
this  environment,  and  to  observe  how  the 
system  responds  in  an  effort  to  achieve 
its  mission.  Points  out  that  the  high¬ 
speed  'digital  computer  makes  this  teehnig,ue 
possible,  for  only  through  its  use  can 
these  complex  environments  be  simulated, 
manipulated,  and  controlled.  Concludes 
that,  on  the  basis  of  experience  with  TATC, 
significant  system  phenomena  can  be  studied 
in  the  laboratory  thereby  contributing 
to  the  development  of  a  science  of  systems. 


UNCLASSIFIED 


UNCLASSIFIED 


UNCLASSIFIED 


